The purpose of the invention is to elicit unbiased forecasts from a plurality of forecasters by means of monetary or other compensation. Each individual forecaster is compensated on the basis of his contribution to the accuracy of a collective prediction, which is computed from a plurality of predictions by individual forecasters. The method may be applied to obtain nearly unbiased estimates or predictions of any variable whose value is currently unknown. Examples of important variables whose value may require forecasting include the expected value of a firm's future profits, the expected price of a commodity, or the expected damage that might be caused by an environmental pollutant.
The invention is particularly focused on efficiently organizing the cost side of information collection, and even more particularly on efficiently motivating and aggregating the predictions of different forecasters. The information being collected is information about the predictions of different forecasters. By the usual law of diminishing marginal returns, after some minimum amount of information has been collected, further information collection will eventually yield decreasing marginal benefits. The optimal amount of information collection occurs when the marginal benefit equals the marginal cost within this range of decreasing marginal benefit, unless the optimal amount is zero.
Each forecaster must make his own subjective prediction of future events based on his own interpretation of objective evidence and his own evaluation of competing hypotheses. It is reasonable to suppose that each forecaster has an important contribution to make to the accuracy of the collective prediction of a group of forecasters. The simplest method of aggregating predictions is to take a mean of the individual predictions. This allows each forecaster to make his own contribution to the outcome of the collective prediction. If each submitted forecast is equally good in terms of expected accuracy, an unweighted mean is best. If some of the forecaster are better than others, a weighted mean is better. Other methods of aggregation include taking a median or computing a trimmed mean. Obviously, the principle of taking means or otherwise aggregating numbers is not new. What is new will be the method of compensating the forecasters.
How forecasters are compensated makes a considerable difference in terms of their incentive to make accurate predictions. For example, suppose we reward forecasters for predicting close to the actual realized value of a variable and penalize forecasters for predicting far away from the actual value. This would seem the most natural way of motivating forecasters to be accurate. "Proper scoring rules" which motivate risk-neutral forecasters to provide unbiased forecasts are basically an elaboration on this intuitive approach. However, when forecasters are risk averse, the use of proper scoring rules can result in biased forecasts.
Instead, the best way of motivating forecasters to make unbiased predictions is to compare each individual forecast with the collective forecast and see whether the individual prediction has moved the collective prediction towards or away from the actual value of the variable being predicted. If the individual prediction has moved the collective prediction towards the actual value, then the forecaster has made a positive contribution and should be rewarded. If the individual prediction has moved the collective prediction away from the actual value, then the forecaster has made a negative contribution and should be penalized. If desired, the rewards/penalties to the forecaster can be made proportional to the estimated marginal benefits/harms caused by the forecaster's prediction.